Pragmatic fidelity measurement in youth service settings.

Background: Fidelity measurement is critical for developing, evaluating, and implementing evidence-based treatments (EBTs). However, traditional fidelity measurement tools are often not feasible for community-based settings. We developed a short fidelity rating form for the Collaborative Problem Solving (CPS) approach from an existing manualized coding system that requires extensive training. We examined the reliability and accuracy of this short form when completed by trained observers, untrained observers, and self-reporting providers to evaluate multiple options for reducing barriers to fidelity measurement in community-based settings. Methods: Community-based treatment providers submitted recordings of youth service sessions in which they did, or did not, use CPS. For 60 recordings, we compared short-form fidelity ratings assigned by trained observers and untrained observers to those provided by trained observers on the manualized coding system. For 141 recordings, we compared providers' self-reported fidelity on the short form to ratings provided by trained observers on the manualized coding system and examined providers' accuracy as a function of their global fidelity. Results & Conclusions: The short form was reliable and accurate for trained observers. An assigned global integrity score and a calculated average of component scores on the short form, but not component scores themselves, were reliable and accurate for observers who had CPS expertise but no specific training on rating CPS fidelity. When providers self-reported fidelity on the short form, their global integrity score was a reliable estimate of their CPS integrity; however, providers with better CPS fidelity were most accurate in their self-reports. We discuss the costs and benefits of these more pragmatic fidelity measurement options in community-based settings.

Developing brief, easy-to-use, and reliable tools to measure how well providers deliver evidence-based treatments (EBTs) in community clinical settings is critical to ensure the benefits of EBTs. However, reliable tools are often too time-consuming and not feasible to use in community settings because they require independent observers to receive intensive training on a coding system and to observe live or recorded treatment sessions for reliable and accurate evaluation. This paper describes steps we took to develop a more practical measure of how well providers deliver one EBT, Collaborative Problem Solving (CPS), based on a previously validated measure, to explore whether the quality of the measure can be maintained while reducing the need for training independent observers and the need for recording treatment sessions. This work contributes to the growing efforts of developing more pragmatic fidelity measures and introduces a new tool, the CPS Practice Integrity Form (CPS-PIF), as a promising measure for community-based clinical settings using CPS.

Evaluating a Modular Approach to Therapy for Children With Anxiety, Depression, Trauma, or Conduct Problems (MATCH) in School-Based Mental Health Care: Study Protocol for a Randomized Controlled Trial.

Introduction: Schools have become a primary setting for providing mental health care to youths in the U.S. School-based interventions have proliferated, but their effects on mental health and academic outcomes remain understudied. In this study we will implement and evaluate the effects of a flexible multidiagnostic treatment called Modular Approach to Therapy for Children with Anxiety, Depression, Trauma, or Conduct Problems (MATCH) on students' mental health and academic outcomes. Methods and Analysis: This is an assessor-blind randomized controlled effectiveness trial conducted across five school districts. School clinicians are randomized to either MATCH or usual care (UC) treatment conditions. The target sample includes 168 youths (ages 7-14) referred for mental health services and presenting with elevated symptoms of anxiety, depression, trauma, and/or conduct problems. Clinicians randomly assigned to MATCH or UC treat the youths who are assigned to them through normal school referral procedures. The project will evaluate the effectiveness of MATCH compared to UC on youths' mental health and school related outcomes and assess whether changes in school outcomes are mediated by changes in youth mental health. Ethics and Dissemination: This study was approved by the Harvard University Institutional Review Board (IRB14-3365). We plan to publish the findings in peer-reviewed journals and present them at academic conferences. Clinical Trial Registration: ClinicalTrials.gov ID: NCT02877875. Registered on August 24, 2016.

Meta-analysis of the effectiveness of the Trier Social Stress Test in eliciting physiological stress responses in children and adolescents.

The Trier Social Stress Test (TSST) is known to reliably induce physiological stress responses in adult samples. Less is known about its effectiveness to elicit these responses in youth samples. We performed a meta-analysis of stress responses to the TSST in youth participants. Fifty-seven studies were included representing 5026 youth participants. Results indicated that the TSST was effective at eliciting stress responses for salivary cortisol (sCort; effect size [ES] = 0.47, p = 0.006), heart rate (HR; ES = 0.89, p < 0.001), pre-ejection period (PEP; ES = -0.37, p < 0.001), heart rate variability (HRV; ES = -0.33, p = 0.028), and systolic blood pressure (ES = 1.17, p < 0.001), as well as negative affect (ES = 0.57, p = 0.004) and subjective anxiety (ES = 0.80, p = 0.004) in youth samples. Cardiac output (ES = 0.15, p = 0.164), respiratory sinus arrhythmia (ES = -0.10, p = 0.064), and diastolic blood pressure (ES = 2.36, p = 0.072) did not reach statistical significance. Overall, effect sizes for the TSST varied based on the physiological marker used. In addition, several physiological markers demonstrated variance in reactivity by youth age (sCort, HR, HRV, and PEP), gender (sCort), type of sample (i.e., clinical versus community sample; sCort and HR), duration of TSST (sCort, HR, HRV, negative affect, and subjective anxiety), number of judges present in TSST (HR and subjective anxiety), gender of judges (sCort), and time of day the marker was assessed (morning versus afternoon/evening; sCort). Overall, the findings provide support for the validity of the TSST as a psychosocial stressor for inducing physiological and psychological stress responses in children and adolescents, but also highlight that some markers may capture the stress response more effectively than others.

Efficient Monitoring of Treatment Response during Youth Psychotherapy: The Behavior and Feelings Survey.

An emerging trend in youth psychotherapy is measurement-based care (MBC): treatment guided by frequent measurement of client response, with ongoing feedback to the treating clinician. MBC is especially needed for treatment that addresses internalizing and externalizing problems, which are common among treatment-seeking youths. A very brief measure is needed, for frequent administration, generating both youth- and caregiver-reports, meeting psychometric standards, and available at no cost. We developed such a measure to monitor youth response during psychotherapy for internalizing and externalizing problems. Across 4 studies, we used ethnically diverse, clinically relevant samples of caregivers and youths ages 7-15 to develop and test the Behavior and Feelings Survey (BFS). In Study 1, candidate items identified by outpatient youths and their caregivers were examined via an MTurk survey, with item response theory methods used to eliminate misfitting items. Studies 2-4 used separate clinical samples of youths and their caregivers to finalize the 12-item BFS (6 internalizing and 6 externalizing items), examine its psychometric properties, and assess its performance in monitoring progress during psychotherapy. The BFS showed robust factor structure, internal consistency, test-retest reliability, convergent and discriminant validity in relation to three well-established symptom measures, and slopes of change indicating efficacy in monitoring treatment progress during therapy. The BFS is a brief, free youth- and caregiver-report measure of internalizing and externalizing problems, with psychometric evidence supporting its use for MBC in clinical and research contexts.

Parental Autonomy Support in Two Cultures: The Moderating Effects of Adolescents' Self-Construals.

Parental autonomy support has been related to positive adolescent outcomes, however, its relation to outcomes in collectivist cultural groups is unclear. This study examined relations of specific autonomy supportive behaviors and outcomes among 401 adolescents (Mage  = 12.87) from the United States (N = 245) and collectivist-oriented Ghana (N = 156). It also examined whether adolescents' self-construals moderated the relations of specific types of autonomy support with outcomes. Factor analyses indicated two types of autonomy support: perspective taking/open exchange and allowance of decision making/choice. In both countries, perspective taking/open exchange predicted positive outcomes, but decision making/choice only did so in the United States. With regard to moderation, the more independent adolescents' self-construals, the stronger the relations of decision making/choice to parental controllingness and school engagement.

When the torch is passed, does the flame still burn? Testing a "train the supervisor" model for the Child STEPs treatment program.

OBJECTIVE: We assessed sustainability of an empirically supported, transdiagnostic youth psychotherapy program when therapist supervision was shifted from external experts to internal clinic staff. METHOD: One hundred sixty-eight youths, aged 6-15 years, 59.5% male, 85.1% Caucasian, were treated for anxiety, depression, traumatic stress, or conduct problems by clinicians employed in community mental health clinics. In Phase 1 (2.7 years), 1 group of clinicians, the Sustain group, received training in Child STEPs (a modular transdiagnostic treatment + weekly feedback on youth response) and treated clinic-referred youths, guided by weekly supervision from external STEPs experts. In Phase 2 (2.9 years), Sustain clinicians treated additional youths but with supervision by clinic staff who had been trained to supervise STEPs. Also in Phase 2, a new group, External Supervision clinicians, received training and supervision from external STEPs experts and treated referred youths. Phase 2 youths were randomized to Sustain or External Supervision clinicians. Groups were compared on 3 therapist fidelity measures and 14 clinical outcome measures. RESULTS: Sustain clinicians maintained their previous levels of fidelity and youth outcomes after switching from external to internal supervision; and in Phase 2, the Sustain and External Supervision groups also did not differ on fidelity or youth outcomes. Whereas all 34 group comparisons were nonsignificant, trends with the largest effect sizes showed better clinical outcomes for internal than external supervision. CONCLUSIONS: Implementation of empirically supported transdiagnostic treatment may be sustained when supervision is transferred from external experts to trained clinic staff, potentially enhancing cost-effectiveness and staying power in clinical practice. (PsycINFO Database Record

A Review of Behavioral Observation Coding Approaches for the Trier Social Stress Test for Children.

The Trier Social Stress Test (TSST) has become one of the most widely-used protocols for inducing moderate psychosocial stress in laboratory settings. Observational coding has been used to measure a range of behavioral responses to the TSST including performance, reactions to the task, and markers of stress induced by the task, with clear advantages given increased objectivity of observational measurement over self-report measures. The current review systematically examined all TSST and TSST-related studies with children and adolescents published since the original work of Kirschbaum et al. (1993) to identify behavioral observation coding approaches for the TSST. The search resulted in 29 published articles, dissertations, and master's theses with a wide range of coding approaches used. The take-home finding from the current review is that there is no standard way to code the Trier Social Stress Test for Children (TSST-C), which appears to stem from the uniqueness of investigators' research questions and sample demographics. This lack of standardization prohibits conclusive comparisons between studies and samples. We discuss relevant implications and offer suggestions for future research.